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ABSTRACT 

The need for an empirically defensible means^ of 
sequencing instruction appears, to have been the primary motivator for 
research into learning hierarchies. Rour methods for generating 
candidates for learning hierarchies were reviewed: introspection, 
formal analysis, observation, and statistical "fishing. »» Experimental 
transfer of training studies and statistical studies have been used 
to evaluate posited hierarchies, and significant modifications of 
these standard evaluative methods have been suggested to improve the 
internal validity of research into learning hierarchies. (Author) 
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SEQUENCING LEARNING 
EVENTS IN PE;RF0RMANCE- BASED 
INSTRUCTIONAL SYSTEMS 



David Lynn Passniore 



Belief in the or<^erliness of Nature has been a kernel idea in Western intel- 
lecfual history from Greek tinies to the present. This deeply-rooted belief has 
acted as an organizing concept in American tradition in the physical and natural 
sciences as well as in economics. Education has also been influenced by this 
thinking. If Nature operates in an orderly manner, then it is thought that the 
most effective methods of conducting the teaching/learning enterprise can be 
uncovered using the scientist's tools. The study of education as a science is 
founded on this belief which i.aturally led to the search for a rational method 
for organising schooling. The zeal which characterized this search is evident 
in Phllbrick's (1885) hyperbole: "If 'America devised the best school desk, it 
must go to the ends of the civilized world [p. 58]." 

At the beginning of the twentieth century» re form- conscious America dis- 
covered "scientific management'* with the help of Frederick W. Taylor (1911). 
"Scientific management" principles injected the concept of efficiency into 
American business practices by developing methods for maximizing industrial out- 
puts for a minimum &i inputs and, thereby, minimizing the cost of the enterprise 
and maximizing the return on capital Invested. Callahan (1962. esif . , p. 23 but 
also passim) documented the sptead of applications of "scientific management" 
dogma to household tasks, family, duties, church functions, and, ultimately, 
the process of education. Not only were the most effective instructional methods 
pursued, but the comparative costs of substitutable methods were also weighed. 
The study of the economics of education is rooted in these principles. 
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Armed with tools such as systems theory, expur iro<.'ntal method, educational 
tests, and cost/effectiveness analysis, instructional systems technicians entered 
the educational research arena In the liturgical spirit of the educational effi- 
ciency niovemtint. And the use of instruction?! systems has continued to be of 
cyclical interest' to American educators throughout the twentuieth century, ta- 
terested readers should consult Hambleton (197A) for an analysis of some recent 

V 

large-'ScalG efforts in this area. 

Frantz (197A) and others (Impelliteri & Pinch, 1971; University of Wisconsin, 
1971) documented the recent upsurge of interest in perf ormance-^based, individu- 

4 

alized instructional systems in occupational education. However, perhaps it 
is safe to say that there still exists a lack of coherent and use- 

ful theory to guide developers of per f ormance-basea instructional systems in 
•occupational education. Recently, though, Lawsoh (1974a, 1974b) engineered a 
theoretical instructional design framework in an attempt to resolve this problem. 

The selection of a sequence for presenting learning evonts was one component in 

r 

Lawson's (1974a, p. 55) methodology for the design of per formance-ba^ed instruc- 

tlonal systems. ' ' * 

Gagn§ (1970, Ch. 4) assigned the term "learning hierarchy" to designate the 
set of dependencies among component skills within a learning t^sk and suggested 
that learning hierarchies might define optimal sequences for presenting learning 
events ( see also, Gagne. 1968; Gagne & Bassler, 1963; Gagne, Mayor, Garstens, 
& Paradise, 1962; Gagne & Paradise, 1961; Gagne & Staff. 1965). Gagne's studies 
seem to have crystalliEed' the methodological directions for subsequent efforts 
in learning hierarchy research by instructional systems specialists (White, 1973, 
p. 367). In performance-based instructional systems, precisely-stated educational 
objectives define the component skilLs of learning tasks; the crucial question 
is whether student's mastery of one objective depends upon their mastery of other 
objectives. 
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The purpose of this paper is 'to supplement and explicate Lawson^s instruc- ' 
tional-theoretic framework by synthesUlng netriods available for generating and 
evaluati"R learning hierarchies for performance-based instructional systems. 
A comprehensive list of references to the general proble\a of sequencing learning 
events may be found in Narchant and Passmore (197A); this paper contains a review 
of the methodological papers in this list which are based on the last 10 or 12 ^ 
ye^rs of research on Gagne's conception of learning hierarchies. The usefulness 
of this paper rests on the. assumption thai it .may be more appropriate tb sequence 
instruction by j^ttending to functional dependencies among objectives in a leaning 
hierarchy thai it 'is to use logical, intuitive sequencing viewpoints (Cf.. how- 
ever, Briggs, 1967, for other sequencing viewpoints). 

jr 

Generation of Candidates for Learninj^ Hierarchies 
A provisional order ihg of instructional objectives mu.it be posited before 

learning hierarchy research can proceed for a particular instructional system. 

How may these candidates for learning hierarchies be genearated? Each^ researcher 

uses some method to complete this task even though the range of available methods 
has not been well documented". And it could be disastrous to operate wltnout a 

hunch about the starting point for this task. Consider that -there are k! pos- 
sible linear orderin.gs for k objectives and that the number of conceivable hier- 
archies capable of being generated from k objectives is much larger. The sources 
of hunches about plausible hierarchies i]iay be grouped into four categories: 
(1) introspection; (2) formal analysis; (3) observation; and (A) statistical 
"fishing'-. 

rntr^spection. One popular method for generating candidates for learning 
hierarchies is to ask thc^quest ion. "What would an individual need to kno-; or 
,do to display competence in this subject matter?" This same question, paraphrased 
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from Gagne and Paradise (1961, p. 4), is applied again to the original answer 
and. then, successively to subsequent answers. This method of questioning pro- 
duces a learning hierarchy characterized by general, molar behaviors at the top 
of the hierarchy and more specific, subordinate, f^nd molecular behaviors near 
the bottoii of the hierarchy (e.g., Gagne, Mayor, Garstens, & Paradise, 19^2, 
p. A). 

This introspective method of hierarchy generation most frequently appears 
in the literatvfre with the instructional system developer the most prominent 
actor in this process. Variations on this theme include the use of subject 
litter experts (Payton, 1971) and students (Kaplan, 1964) to play introspective 
roles to generate learning hierarchies. 

F orma l anal ysis. Formal analyses of subject matter domains could provide 
another source of learning hierarchies. Good examples of such formal analyses 
are the industrial manufacturing and construction domains developed by the Indus- 
trial Arts Curriculum Project (Towers, Lujc, & Ray, 1966). F^ch of these subject 
matter domains was organized as a hierarchical taxonomy. Levels of each hierarchy 
proceed from general to more specif ic elements of manufacturing and cdnstructloa 
practice. ^Within each levels elements were devised to be individually inclusive 
and mutually exclusive- 
Hierarchies supplied by the introspective method previously d;iscussed are 
developed by imagining the learner's interaction with the subject matter. In 
contrast, .the method of formal analysis removes consideration of- the learner and, 
instead, applies the logical rigor of taxonomlc inquiry that has been developed 
for the natural sciences (see Greg'g, 1954). 

Observation . Learning hierarchies could al&o be posited by observing the 
natural order in which learners acquire behaviors. For exara'^le, a problem could 
be assigned to learners and, then, the milestones in their pre ress toward pro- 
blem solution could be recorded. Perhaps observational methods similar to those 
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described by Placet and Inhelder (1969) for researching hierarchies airong devel- 
opment tasks might be fruitful in learning hierarchy research. 

-While the method of observation introduces a realistic glimpse at learners 
as a source of learning hierarchies, it must be remeipbered that the sequence 
which learners c^ioose to solve a problem may prove to be neither effective nor 
efficient. And maximum eXf ect iveness and cf lency in facilitating learning^ 
are primary quality criteria for instructional systems. 

Statistical"fishin8 ." Also, candidates for learning hierarchies are often 
captured through statistical studies which may be best characterized as f is -ing 
expeditions. The prior absence of a hypothesized network of educational objec- 
tives i§ a distinguishing feature of these types of studies. The main aim of 
these statistical studies is to suggest empirically-based, post hoc, structural 
hypotheses about the relationships among a set of behaviors. 

Many of the numerical techniques, such as hierarchical cluster analysis 
(e.g.. Tryon, 1958) and hierarchical ' factor analysis (e.g.. Coombs & Satter, 
19A9), applied to military task analysis could be used to fish for provisional ^ 
candidates for learning hierarchies. A conference report by Moss and Smith (1970, 
esp. papers by Ammerman. Christal, and Silverman) described the application of 
military task analysis techniques to the construction of vocational education < 

» 

curricula. 

Baker (1972), Cnenzoff (196A), FoUey (1964), Morsh, Madden, and Christal 
(1961) reviewed some of the more standard numerical approaches to the development 
of hierarchical taxonomies of behavior! In addition, a rich and suggestive source, 
of numerical taxonomy methods that could. be applied in learning hierarchy research 
may be found in the natural science literature (For starters see, Sokal & Sneath, 
1963"^or Sneath & Sokal, 1962). Driver's (1963) survey of numerical clasifica- 
tion methods in anthropology might a^so be useful. 

Another prominent line of thought in learning hierarchy research is that 
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functional dependencies araong' instruc t ional objectives can be discovered by 
examining the Guttnian (194A) scalability of the test items measuring the be- 
haviors specified by the objectives. Since the success of Guttman's procedure 
demands a linear ordefinglfef items which measure a un Idlroensional entity, some 
researchers have preferred to use Lingoes (1963) multidimensional extension 
of Guttman's procedure to uncover non-linear orderings araong educational .objfec- 

'ves. Alrasian and 'Bart (1971) and Bart and Kruz (1973) developed a similar 
but more parsimonious technique for discovering hierarchies among test items. 
Applications of the Bart et^ a_l. ordering-theoretic method were presented in ^ 
Alrasian and Bart (1974), Bart (1972a, 1972b) and Bart and Alrasian (1972). 

Evaluating Posited Hierarchies 
An ordering of objectives posited for an instructional system roust be 
evaluated to determi,ne whether it should be rejected, modified, or adopted. 
Experimental transfer of training studies as well aa statistical studies have 
been used in this evaluation task. Both types of studies were extensively re- 
viewed and . heavily criticized by White (1973, 1974a, •1974b) t;ho, as a result, 
presented (White, 197Ac^, White & Clark, 1973) significant modifications of 
earlier paradigms for research into learning hierarchies. , ♦ 

St andar d evaluat ion methods . A transfer of training effect is the action 
that learning one task has upon subsequent learning or performance of another 
task (Andreas, i972, p. A?9). Networks of positive — that is, benefirlal and 
facilitating — transfer among a set of instructional objectives are frequently 
sought in the evaluation of learning hierarchies for instructional system design. 
The definition of a hierarchical relationship ns positive transfer among learning 
events was used in Gagne's pioneering studies which set the tone for moet of the 
learning hierarchy research efforts that followed. 

How might mastery of one instructional objective facilitate the m&stery of 
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another objective? Several hypothf\«^efi arc reasonable. First, some identical 
behaviors may be required to master both objectives. iJork habits or specific 
subject matter skills arc cx.itnples of identical behaviors that might precipitate 
transfer effects. Second, positive transfer tnay occur when similar stimuli and 
responses are involved in the mastery of both objectives. The psychological 
mechanisms operating to Induce transfer in this case would be stimulus and 
response gtMieral ieat Ion » 

Transfer of training studies are usually .conducted in controlled settings. 
In the application of such studies to learning hierarchy research, factors^ such 
as warm-up effects and transfer of general work habits must be partitioned from 

r 

the transfer of specific subject matter skills to unequivocally answer the ques-- 
tion^'^What is learned?". Murdock's (1957) evaluative review of numerous designs 
used in transfer experiments might be helpful to learning hierarchy researchers 
as may the review and critique of methods for measuring transfe^r effects pro- 
vided by Gagne, Foster, and Crowley (1948). 

. In addition to experimental studies, the same methods described as methods 
for statistical '•fishing'' for learning hierarchy candidates are alsp applied 
in the evaluation of posited hierarchies. Rather*than using these statistical 
tecliniques to hypoth^sl^e hierarchies,' the focus, instead^ is on testing hypo-- 
thesi ^cd 'hierar<?h ies. 

An interest in validating loarivlng hierarchies not only implies that infor- , 
mat ion will be obtained to discern the appropriateness of the posited hierarchies^ 
but also that the hierarchies will be improved if necessary. However, the range 
of decisions and* decision-making strategies, for improving learning hierarchies 
have not been carefully delineated. For example, one decision that could be made 
on the basis of experimental or statistical evaluative evidence is that the objec- 
tives need to be reordered. Another decision that could be made based on the same 
data might be that the objectives need to be subdivided^ refined, and, then. 



10 



.reordered. An explicit method for rationally choosing between these com- 

« 

PV'ting decisions is lacking but could be a fruitful line of future research.. 

White's modif icatiohs . White (1973, p. 371; 1974a. p.l) felt that most 
evaluations' of learning hierarchies were hindered by the following six pro- 
b 1 ems : ' , 

' • V 

(1) small sample s\zei\ . \ 

<2) imprecise specification of hierarchy elements; 

(3) the use of only one question per element to test the dependencies 
among elements; 

(4) the absence of a test of hierarchical independence that takes error 
* of measurement i nto, account; 

(5) the practice of testing for dep'endencies among hierarchy elements 
after instruction is completed on all elements, -thus confounding 
forgetting with a lack of connection between eleroentfs; 

and (6) lack of face validity for hierarchies which have been empirically 
evaluated and improved. 

White's solution to the f irst- problem was to increase the size of the sam- 
ple bf subjects selected for learning ^lierarchy evaluations. White suggested 
that problem two could be best solved through more careful introspection during 
the generation of a learning hierarchy candidate. However, no clues were given 
on how to detect this problem during the evaluation phase of learning hierarchy 
research. 

Problems three and four are related. The use of only one\uestion per 
element does not allow the estimation of the error in measuring the dependency 
among elements. Moreover, none of the existing dependency indexes reviewed by 
C;ipie and Jones (1971) make the concept of error of measurement explicit. Also 
White (1974b) charged that indexes such as Gagne and Paradise's (1961) propor-. 
tion of positive transfer as well as varients of it proposed by Walbesser and 
Kisenberg (1972). Guttcian's coefficient of reproducibility (used ^in Resnick & 
Wang, 1969), and the^' four-fold contingency table and related phi . correlation 
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coefficient su^;gcstt-ci by Capi.- and .lones (1971) share a cutnmon probU-r..: ... u ^ 

of these indexes" can have values that indicate a hierarchical conm-c 1 1 • ^ 

r 

when the skills are rtMlly independent. i 

White and Clark (1973) iniroducPd a statistical test oi h i erarc h i . d i ^ 
dency which trhey purport treats tht- prohletn of error or 'measurcroent .-ind also 
allows the .Inference af hierarchical dependencies among ob j ec. i i.vt-s tor r.r ; . 
ulation of interest." White and Clark .'arofuUy discussed the pouer fuv.- t^.r. .-r 
this signifirance test in iheir i^iiXcli^cMn_ej_riJ<a papu-r. Unfortunately. U^n i •. e :a'. 
tn discuss power issues in subset^uent articles alned at U-^fi technicallv --fo, ; 
ticated audiences. And it is pr^'oisely these audiences which repeatfdly tai) 
to recognize relationships b^^tween sample size and the power of significance 
tests in their .research work (see Mcehl. 1967). Consequently, educat i cn.i 1 ly 
valid connections anonj.; elements in a hierarchy may. and probahly will, be re- - 
jected if standard Neywan-Fearson hypothesis test ing. procedures are used ^ ^.xper- 
iruents with large samples of subjects that White also suggested. This point r.usr 
be explained to practical tesearcli workers. 

• To solve the fifth problem, White advocated the administration of tes". iteas 
at key points during a learning program designed to teach the subject matter in 
the. hypothesized s'^quence under consideration. Tlien, White and Clark's signi- 
ficance test may be applied to determine whether dependencies exists among ele- 
ments in a posited hierarchy. Of.coorse, this raises a fidelity Issue: Does 
i^-Riake a difference that the hierarchy under consideration may not ultimately 
be used to sequence a learning program? ■ Perhaps* this question merits . invest i- 
gat ion. 

The statement of Whitens sAXth problem seen-s to be ^ reaction to blind, 
"dustbowl" empiricism in learning hierarchy research. White cLai.i.ed that learning 
hierarchies modified on the basis of evaluative information should also be re- 
quired to make logical, intuitive sense. If this is so, then what purpose does 
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ihe evaluaticui serv** and how are "doc is ions to be made with an appropriate mix 
of logical and empirical information? On the other hand, unexamined learning 
f; it-rarchius nay hnvo an intuitive appeal but nay also sc-^riously lack empirical 
ir.:;'ort. As has already hv-on asserted in this paper» the explieation of decisions 
ant! decision-mking strategies for the iniprovement of learning hierarchies would 
be a nt^teworthy contr ihvu i on to the liter^iture. 

Concluding Remarks 

The neecf for an empirically defensible means of sequencing instruction ap- 
pears to have been the primary inoiivator for research into learning hierarchies. 
Fviir methods for gt»nerating candi^^ates for learning hierarchies were reviewed; 
int D-spect ion, formal analytJis, observation, and statistical "fishing". Exper- 
imental transfer of training litudies and statistical studies have been used co 
evaluate posited hierarchies and significant modifications of these standard 
evaluative methods have been suggested to improve the internal validity of re- 
^;ea^ch inta^' learning hierarchies. ^ 

Would the stu<iy of learning hierarchies have been important if a researcher 

wilii Gign^'s Ntalure had not chosen t.-^ become i-volved? Of course, an equally 

valid question is whether learning hierarchy research would have been disco--^ 

vered or received any creative impetus without his Involvement? More ^generally, 

» 

art- topics in instructional science resi'arched because of the importance of their 
p.itrcns or .ire they studied for their intrinsic importance? Unraveling the an- ^ 
. -^ to tiu sK qut ijtions may .sc-rve to di iwyst if y tl>e study of learning hierarchies 
;<:i(l li-.id to the statenit^nt of several criticai questions for the future of learning 
hit-r.irc.hy research. 

Could we trim away our interest in learning hierarchies and, thereby* achieve 
a more parsimonious instructional science? Could we conduct the teaching/ learning 
.enterprise without learning hiterarchies? Would any instructional sc-quence be 
bi'tter than none at least as good as one suggested by resource-consuming learning 

S 
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hierarchy research"? - Answers to these crucial questions, and otherb» may con- 
tribute evidence for determining the external validity or Learning hierarchy 
'studies. Perhaps, these-squest ions are ju?t as important, is the present exclu- 
sive and pervasive interest in the internal validity of learning hierarchy ex- 
periments which has been reviewed in this paper.' 
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